Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rabbit] enable quorum queues #884

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

SeanMooney
Copy link
Contributor

@SeanMooney SeanMooney commented Oct 25, 2024

This change simplfies the oslo_messaging_rabbit config generations
by enabling rabbit_quorum_queue and hardcoding heartbeat_in_pthread
as false.

Closes: OSPRH-9736
Related: OSPRH-10790

@openshift-ci openshift-ci bot requested review from abays and viroel October 25, 2024 10:28
Copy link
Contributor

openshift-ci bot commented Oct 25, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: SeanMooney

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@bogdando
Copy link
Contributor

let's test it with downstream uni job that runs minor updates

@SeanMooney SeanMooney force-pushed the quorum_queues branch 2 times, most recently from f95b584 to 97de8d4 Compare October 25, 2024 11:02
@@ -55,19 +55,10 @@ key={{.SSLCertificateKeyFile}}
lock_path = /var/lib/nova/tmp

[oslo_messaging_rabbit]
amqp_durable_queues=false
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rabbit notes that durable queue are required for quorm queues
https://www.rabbitmq.com/docs/quorum-queues#feature-comparison
but the oslo messaging option is ignored if quorum queues are enabled
https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/_drivers/amqp.py#L33-L37
so its more correct to not set this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, good point, this wont' work w/o durable queues, and that requires separated vhosts for osp services, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last time I tested a rabbit cluster with quorum queues with my Jepsen fork, it was not possible to do w/o setting durable queues, FYI.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so i think reading the commnet in oslo that oslo will enable durable queue if you request quorma cues and ignore the amqp_druable_queues value

this failed with

amqp.exceptions.PreconditionFailed: Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'nova' in vhost '/': received 'true' but current is 'false'

and i think that was because all the other operators are using false.

ill try expclity setting ti to true but i think this just wont work without vhost

Copy link
Contributor

@bogdando bogdando Oct 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this just wont work without vhost

yes, as other services would conflict on creating things (like exchange 'openstack') there with different persistence flags

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically, we might want to test the altogether switch of the openstack services to that option in the downstream testproject

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically you should not be doing any dev downstream and we should be testing changing this everywhere upstream.

we could just make nova use a separate rabbit for this testing but in reality this is better to test manually in a local deployment then try and do it in the ci.

with that said i wont really have time to work on this so what ever way works for you.

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/d7f967eea0394f49b1a180bc92d12552

✔️ openstack-meta-content-provider SUCCESS in 1h 28m 02s
✔️ nova-operator-kuttl SUCCESS in 42m 08s
nova-operator-tempest-multinode FAILURE in 1h 10m 25s
nova-operator-tempest-multinode-ceph FAILURE in 1h 11m 30s

This change simplfies the oslo_messaging_rabbit config generations
by enabling rabbit_quorum_queue and hardcoding heartbeat_in_pthread
as false.

Closes: OSPRH-9736
Related: OSPRH-10790
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/3f14e91efe7f462c81a155dc27155d70

✔️ openstack-meta-content-provider SUCCESS in 1h 27m 29s
nova-operator-kuttl FAILURE in 50m 53s
nova-operator-tempest-multinode FAILURE in 1h 11m 33s
nova-operator-tempest-multinode-ceph FAILURE in 1h 11m 18s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants